Ultrasound observation apparatus, method of operating ultrasound observation apparatus, and computer readable recording medium

ABSTRACT

An ultrasound observation apparatus for generating an ultrasound image based on an ultrasound echo acquired by an ultrasound probe provided with an ultrasound transducer includes: a puncture needle detection unit configured to detect an image of a puncture needle displayed in the ultrasound image; a motion extraction unit configured to extract a linear motion at a point of the puncture needle based on a history of the image of the puncture needle in the ultrasound image; and a composite image generation unit configured to generate a composite image by generating loci of the linear motions extracted by the motion extraction unit and superimposing the loci on the ultrasound image, the composite image generation unit generating the composite image by using the loci of the linear motions having passed through a common section a plurality of times among the loci of the linear motions extracted by the motion extraction unit.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of PCT international application Ser.No. PCT/JP2016/060573, filed on Mar. 30, 2016 which designates theUnited States, incorporated herein by reference, and which claims thebenefit of priority from Japanese Patent Applications No. 2015-133871,filed on Jul. 2, 2015, incorporated herein by reference.

BACKGROUND

The present disclosure relates to an ultrasound observation apparatusfor observing a tissue to be observed, using ultrasound waves, a methodof operating an ultrasound observation apparatus, and an ultrasoundobservation apparatus operation program.

Needle biopsy, which is performed using a puncture needle whendiagnosing a tissue to be observed using ultrasound waves, has a verynarrow range of tissues that can be collected by one puncture motion andhas a small amount of tissues that can be collected. To collect a widerange of tissues, the needle biopsy may be performed a plurality oftimes at different positions on the same cross section. Further, toincrease the collection amount of the tissues, the puncture needle maybe reciprocated a plurality of times at the same position.

When performing the needle biopsy a plurality of times, it is not easyto grasp the position where the needle biopsy has been performed afterthe puncture needle has been taken out from the tissue to be observed.To solve this problem, conventionally, a technology of storing aposition that the puncture needle has reached within a tissue anddisplaying the position on a monitor is disclosed (see, for example, JP2009-189500 A).

SUMMARY

An ultrasound observation apparatus according to one aspect of thepresent disclosure generates an ultrasound image based on an ultrasoundecho acquired by an ultrasound probe provided with an ultrasoundtransducer that transmits an ultrasound wave to an observation targetand receives an ultrasound wave reflected at the observation target, andincludes: a puncture needle detection unit configured to detect an imageof a puncture needle displayed in the ultrasound image; a motionextraction unit configured to extract a linear motion at a point of thepuncture needle based on a history of the image of the puncture needlein the ultrasound image; and a composite image generation unitconfigured to generate a composite image by generating loci of thelinear motions extracted by the motion extraction unit and superimposingthe loci on the ultrasound image, the composite image generation unitgenerating the composite image by using the loci of the linear motionshaving passed through a common section a plurality of times among theloci of the linear motions extracted by the motion extraction unit.

Moreover, an ultrasound observation apparatus according to one aspect ofthe present disclosure generates an ultrasound image based on anultrasound echo acquired by an ultrasound probe provided with anultrasound transducer that transmits an ultrasound wave to anobservation target and receives an ultrasound wave reflected at theobservation target, and includes: a puncture needle detection unitconfigured to detect an image of a puncture needle displayed in theultrasound image; a motion extraction unit configured to extract alinear motion at a point of the puncture needle based on a history ofthe image of the puncture needle in the ultrasound image; and acomposite image generation unit configured to generate a composite imageby generating loci of the linear motions extracted by the motionextraction unit and superimposing the loci on the ultrasound image, thecomposite image generation unit generating a composite image displayingloci of the linear motions during time from when the puncture needledetection unit detects the image of the puncture needle to when thepuncture needle detection unit stops detecting the image of the punctureneedle, and loci of the linear motions during time from when thepuncture needle detection unit detects the image of the puncture needleagain to when the puncture needle detection unit stops detecting theimage of the puncture needle, in different display forms from eachother.

The above and other features, advantages and technical and industrialsignificance of this disclosure will be better understood by reading thefollowing detailed description of presently preferred embodiments of thedisclosure, when considered in connection with the accompanyingdrawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram schematically illustrating a configuration of anultrasound diagnosis system including an ultrasound observationapparatus according to a first embodiment;

FIG. 2 is a perspective view schematically illustrating a configurationof a distal end portion of an insertion portion and a distal end of arigid portion of an ultrasound endoscope;

FIG. 3 is a block diagram illustrating functional configurations of theultrasound observation apparatus according to the first embodiment anddevices connected to the ultrasound observation apparatus;

FIG. 4 is a diagram schematically illustrating a state in which an imageof a puncture needle is displayed on a B-mode image;

FIG. 5 is a diagram schematically illustrating a display example of acomposite image generated by a composite image generation unit of theultrasound observation apparatus according to the first embodiment;

FIG. 6 is a flowchart illustrating an outline of processing performed bythe ultrasound observation apparatus according to the first embodiment;

FIG. 7 is a diagram schematically illustrating a display example of acomposite image in a first modification of the first embodiment;

FIG. 8 is a diagram schematically illustrating a display example of acomposite image in a second modification of the first embodiment;

FIG. 9 is a diagram schematically illustrating a display example of acomposite image in a third modification of the first embodiment;

FIG. 10 is a flowchart illustrating an outline of processing performedby an ultrasound observation apparatus according to a second embodiment;and

FIG. 11 is a diagram schematically illustrating a display example of acomposite image according to the second embodiment.

DETAILED DESCRIPTION

Hereinafter, embodiments will be described with reference to theaccompanying drawings.

First Embodiment

FIG. 1 is a diagram schematically illustrating a configuration of anultrasound diagnosis system including an ultrasound observationapparatus according to a first embodiment. An ultrasound diagnosissystem 1 illustrated in FIG. 1 includes an ultrasound endoscope 2, anultrasound observation apparatus 3, a camera control unit (CCU) 4, adisplay device 5, and a light source device 6.

The ultrasound endoscope 2 transmits ultrasound waves to a subject thatis an observation target and receives the ultrasound waves reflected atthe subject. The ultrasound endoscope 2 includes a tubular insertionportion 21 that is inserted into the subject, an operating unit 22provided to a proximal end portion of the insertion portion 21 and whichis held by a user and receives an operation input from the user, anuniversal cord 23 extending from the operating unit 22 and including aplurality of signal cables, an optical fiber that transmits illuminationlight generated by the light source device 6, and the like, and aconnector 24 provided on an end portion of the universal cord 23 on anopposite side of the operating unit 22. For example, the ultrasoundendoscope 2 observes digestive tract (esophagus, stomach, duodenum, orlarge intestine) and respiratory tract (trachea or bronchus) of thesubject, as the observation target, and various types are knowndepending on the observation target.

The insertion portion 21 includes a distal end portion 211 in which anultrasound transducer is provided, a rigid portion 212 externallycovered with a rigid member connected to a proximal end side of thedistal end portion 211, a bend portion 213 provided to a proximal endside of the rigid portion 212 and bendable according to the operationinput received by the operating unit 22, and a flexible tube portion 214provided to a proximal end side of the bend portion 213 and externallycovered with a member having flexibility. A treatment tool channel 215that is an insertion passage into which a treatment tool including apuncture needle is inserted is formed inside the insertion portion 21(the treatment instrument channel is illustrated by the broken line inFIG. 1). Further, a light guide that transmits the illumination lightsupplied from the light source device 6 and a plurality of signal cablesthat transmits various signals are provided inside the insertion portion21 (not illustrated).

FIG. 2 is a perspective view schematically illustrating a configurationof the distal end portion 211 of the insertion portion 21 and a distalend of the rigid portion 212. The distal end portion 211 includes aconvex ultrasound transducer 211 a. The ultrasound transducer 211 a maybe an electronic scanning-type transducer or a mechanical scanning-typetransducer. A treatment tool opening portion 212 a communicating withthe treatment tool channel 215, an imaging opening portion 212 bcollecting light from an outside and guiding the light to an imagingoptical system, an illumination opening portion 212 c located at adistal end side of the light guide and which emits the illuminationlight, and an air and water feed nozzle 212 d are provided in a distalend of the rigid portion 212. The treatment tool opening portion 212 ais provided with a rising base 212 e that allows the treatment tool tobe placed thereon in a manner that a protruding direction of theinstrument tool to an outside is changeable. The rising base 212 e iscapable of changing a rising angle by an operation input of theoperating unit 22. An objective lens is attached to the imaging openingportion 212 b and an illumination lens is attached to the illuminationopening portion 212 c.

FIG. 2 illustrates a state in which a puncture needle 100, which is akind of the treatment tool, protrudes from the treatment tool openingportion 212 a. The puncture needle 100 is inserted into the treatmenttool channel 215 via a treatment tool insertion port 221 (see FIG. 1)formed in the operating unit 22, and protrudes to an outside through thetreatment tool opening portion 212 a.

The configuration of the ultrasound diagnosis system 1 is continuouslydescribed with reference to FIG. 1. The connector 24 connects theultrasound observation apparatus 3, the camera control unit 4, and thelight source device 6. From the connector 24, an ultrasound cable thattransmits and receives a signal to and from the ultrasound observationapparatus 3, an electrical cable that transmits and receives a signal toand from the camera control unit 4, and the light guide that transmitsthe illumination light generated by the light source device 6 extend. Anultrasound connector connected to the ultrasound observation apparatus 3is provided in a distal end of the ultrasound cable. An electricalconnector connected to the camera control unit 4 is provided in a distalend of the electrical cable.

The ultrasound observation apparatus 3 transmits and receives anelectrical signal to and from the ultrasound endoscope 2 via theultrasound cable. The ultrasound observation apparatus 3 appliespredetermined processing to an electrical echo signal received from theultrasound endoscope 2 to generate an ultrasound image or the like.Details of the function and configuration of the ultrasound observationapparatus 3 will be described below with reference to the block diagramof FIG. 3.

The camera control unit 4 applies predetermined processing to an imagesignal received from the ultrasound endoscope 2 through the electricalcable to generate an endoscope image.

The display device 5 is configured using liquid crystal, organic electroluminescence (EL), or the like, and receives data of the ultrasoundimage generated by the ultrasound observation apparatus 3, the endoscopeimage generated by the camera control unit 4, and the like and displaysthe images.

The light source device 6 generates the illumination light forilluminating an inside of the subject and supplies the illuminationlight to the ultrasound endoscope 2 via the light guide. The lightsource device 6 also incorporates a pump for sending water and air.

FIG. 3 is a block diagram illustrating functional configurations of theultrasound observation apparatus according to the first embodiment anddevices connected to the ultrasound observation apparatus. Theultrasound observation apparatus 3 includes a transmitting and receivingunit 31 that transmits and receives a signal to and from the ultrasoundtransducer 211 a, a signal processing unit 32 that generates digitalreception data based on an echo signal received from the transmittingand receiving unit 31, an input unit 33 realized using a user interfaceof a keyboard, a mouse, and a touch panel, and which receives inputs ofvarious types of information including a motion instruction signal ofthe ultrasound observation apparatus 3, a puncture needle detection unit34 that detects the puncture needle included in the ultrasound image, amotion extraction unit 35 that extracts a linear motion of the punctureneedle based on a history of a position of a point of an image of thepuncture needle in the ultrasound image, an image generation unit 36that generates data of various types of images including the ultrasoundimage, using information of the reception data generated by the signalprocessing unit 32, a control unit 37 that collectively controls theoperation of the entire ultrasound diagnosis system 1, and a storageunit 38 that stores various types of information necessary for theoperation of the ultrasound observation apparatus 3.

The transmitting and receiving unit 31 transmits a pulse transmissiondrive wave signal to the ultrasound transducer 211 a based on apredetermined waveform and transmission timing. Further, thetransmitting and receiving unit 31 receives an electrical echo signalfrom the ultrasound transducer 211 a. The transmitting and receivingunit 31 also has functions to transmit various control signals outputtedby the control unit 37 to the ultrasound endoscope 2, and receivevarious types of information including an ID for identification from theultrasound endoscope 2 and transmit the information to the control unit37.

The signal processing unit 32 applies known processing such as band-passfilter, envelope detection, and logarithmic conversion to the echosignal to generate digital ultrasound image reception data, and outputsthe generated data. The signal processing unit 32 is realized using ageneral-purpose processor such as a central processing unit (CPU) or adedicated integrated circuit or the like that executes a specificfunction such as application specific integrated circuit (ASIC) or fieldprogrammable gate array (FPGA).

The puncture needle detection unit 34 detects the puncture needledisplayed in the ultrasound image by image processing, and writes andstores coordinates of the position of the point of the detected punctureneedle in the ultrasound image together with information of detectiontime to a puncture needle information storage unit 381 included in thestorage unit 38. The puncture needle detection unit 34 detects, forexample, a region having a large luminance value as the puncture needleby analyzing the luminance of pixels of the ultrasound image. Note thatthe puncture needle detection unit 34 may detect the puncture needle byperforming pattern matching using the ultrasound image of the punctureneedle stored in advance by the storage unit 38.

The motion extraction unit 35 extracts a motion that forms a linearlocus in the ultrasound image based on the puncture needle informationstored in the puncture needle information storage unit 381. At thistime, the motion extraction unit 35 extracts movement of the punctureneedle from the start to the end in the same direction as a singlemotion. By extracting the motion that forms a linear locus by the motionextraction unit 35 in this way, a locus of when changing a direction topuncture by the puncture needle 100 by changing the rising angle of therising base 212 e is deleted, for example. Note that “linear” referredto here includes not only the case where the point of the punctureneedle is moved on one straight line in the ultrasound image but also acase where the point of the puncture needle is moved in a long andnarrow rectangular region having a small width set in advance in adirection orthogonal to a moving direction along the straight line.

The image generation unit 36 includes an ultrasound image generationunit 361 that generates ultrasound image data based on the receptiondata, and a composite image generation unit 362 that generates acomposite image by generating the locus of the linear motion of thepuncture needle extracted by the motion extraction unit 35 andsuperimposing the locus on the ultrasound image.

The ultrasound image generated by the ultrasound image generation unit361 is a B-mode image obtained by converting amplitude into luminance.FIG. 4 is a diagram schematically illustrating a state in which an imageof a puncture needle is displayed in a B-mode image. In a B mode image101 illustrated in FIG. 4, a puncture needle image 111 linearlyextending from an upper right part toward a central part of a screen isdisplayed. Note that, in FIG. 4, concrete display of the ultrasoundimage other than the puncture needle image 111 is omitted. The omissionof concrete image of the ultrasound image other than the puncture needleimage 111 is similarly applied to the composite image to be referred tobelow.

The composite image generation unit 362 generates a composite image bysuperimposing the locus of the linear motion extracted by the motionextraction unit 35 on the ultrasound image. FIG. 5 is a diagramschematically illustrating a display example of a composite imagegenerated by the composite image generation unit 362. In a compositeimage 102 illustrated in FIG. 5, a locus group 121 composed of aplurality of straight lines is displayed. In the first embodiment, thecomposite image generation unit 362 generates a composite image afterthe puncture needle image becomes undetected for the first time afterthe puncture needle detection unit 34 starts detecting the image of thepuncture needle. That is, in the first embodiment, the composite imagegeneration unit 362 generates the composite image after the punctureneedle detection unit 34 stops detecting the image of the punctureneedle due to taking out of the puncture needle from the ultrasoundendoscope 2 after termination of the first needle biopsy.

The composite image generation unit 362 generates the composite image byarranging lines representing the locus of the linear motion extracted bythe motion extraction unit 35 while adjusting a display area of theimage of the puncture needle in accordance with change of the positionof the observation target, of each frame, displayed in the B-mode image.

In a case where the puncture needle detection unit 34 again detects theimage of the puncture needle after the composite image generation unit362 generates the composite image, the composite image generation unit362 stops generation of the composite image. In this case, the B-modeimage is displayed on the display device 5. In this case, the compositeimage generation unit 362 may generate a composite image displaying alocus in a form not disturbing the visibility of the puncture needle.Examples of the form not disturbing the visibility of the punctureneedle include displaying the locus by a broken line and displaying thelocus in a less visible color.

In a case where a freeze button of the operating unit 22 receives anoperation input after the composite image generation unit 362 generatesthe composite image, the composite image generation unit 362 may stopgeneration of the composite image. In this case, when canceling thefreeze upon receipt of an operation input again by the freeze button,the composite image generation unit 362 may resume the generation of thecomposite image.

The control unit 37 includes a display controller 371 that controlsdisplay of the display device 5. The display controller 371 causes thedisplay device 5 to display the various images generated by the imagegeneration unit 36.

The control unit 37 is realized using a general-purpose processor suchas a CPU having arithmetic and control functions, or a dedicatedintegrated circuit such as ASIC or FPGA. In the case where the controlunit 37 is realized by the general-purpose processor or the FPGA, thecontrol unit 37 reads various programs and data stored in the storageunit 38 from the storage unit 38, and executes various types ofarithmetic processing related to the operation of the ultrasoundobservation apparatus 3 to collectively control the ultrasoundobservation apparatus 3. In the case where the control unit 37 isconfigured from the ASIC, the control unit 37 may independently executevarious types of processing, or may execute the various types ofprocessing using various data stored in the storage unit 38. In thefirst embodiment, the control unit 37 and part of the signal processingunit 32, the puncture needle detection unit 34, the motion extractionunit 35, and the image generation unit 36 can be configured from acommon general-purpose processor, a dedicated integrated circuit, or thelike.

The storage unit 38 includes the puncture needle information storageunit 381 that stores information of the point position of the image ofthe puncture needle detected by the puncture needle detection unit 34together with the information of the detection time of the image of thepuncture needle and the like, as puncture needle information. Thepuncture needle information stored in the puncture needle informationstorage unit 381 is deleted under control of the control unit 37 whenthe puncture needle detection unit 34 starts detecting the image of thepuncture needle again after having stopped detecting the image of thepuncture needle.

The storage unit 38 stores various programs including an operationprogram for executing an operation method of the ultrasound observationapparatus 3. The various programs including an operation program canalso be recorded on a computer readable recording medium such as a harddisk, a flash memory, a CD-ROM, a DVD-ROM, or a flexible disk and widelydistributed. Note that the above-described various programs can also beacquired by being downloaded via a communication network. Thecommunication network referred to here is realized by, for example, anexisting public line network, a local area network (LAN), or a wide areanetwork (WAN), and may be a wired or wireless network.

The storage unit 38 is realized using a read only memory (ROM) in whichthe various programs and the like are installed in advance, a randomaccess memory (RAM) in which arithmetic parameters and data ofprocessing, and the like are stored, and the like.

FIG. 6 is a flowchart illustrating an outline of processing performed bythe ultrasound observation apparatus 3. The flowchart illustrated inFIG. 6 illustrates processing after the transmitting and receiving unit31 starts transmission of a transmission drive wave according to anobservation mode, and the ultrasound transducer 211 a startstransmission of an ultrasound wave.

First, the transmitting and receiving unit 31 receives an echo signalthat is a measurement result of an observation target by the ultrasoundtransducer 211 a from the ultrasound endoscope 2 (Step S1).

Next, the transmitting and receiving unit 31 applies predeterminedreception processing to the echo signal received from the ultrasoundtransducer 211 a (Step S2). To be specific, the transmitting andreceiving unit 31 amplifies (STC correction) the echo signal and thenapplies processing such as filtering or A/D conversion to the amplifiedsignal.

After that, the ultrasound image generation unit 361 generates a B-modeimage, using the echo signal processed by the transmitting and receivingunit 31, and outputs data of the B-mode image to the display device 5(Step S3).

The puncture needle detection unit 34 performs processing of detectingthe image of the puncture needle displayed in the B-mode image, usingthe generated B-mode image (Step S4). When the puncture needle detectionunit 34 detects the image of the puncture needle in the B mode image(Step S4: Yes), and when the composite image generation unit 362 hasgenerated a composite image in a previous frame (Step S5: Yes), theultrasound observation apparatus 3 proceeds to Step S6.

In Step S6, the display controller 371 deletes (non-displays) the locusor changes the display method (Step S6). The puncture needle detectionunit 34 writes and stores the information of the point position of theimage of the detected puncture needle together with the information ofdetection time and the like to the puncture needle information storageunit 381 (Step S7). The information of the point position of the imageof the puncture needle is represented by coordinates in the B-modeimage, for example. Step S6 corresponds to a situation in which thepuncture needle is newly inserted while the display device 5 isdisplaying the composite image. In Step S6, the locus of the previouspuncture needle may be deleted, or the display method may be changed sothat the locus can be identified as the previous locus.

Next, the display controller 371 performs control to cause the displaydevice 5 to display the B-mode image (Step S8).

After that, when the input unit 33 receives an input of a signalinstructing termination (Step S9: Yes), the ultrasound observationapparatus 3 terminates the series of processing. On the other hand, whenthe input unit 33 does not receive the input of a signal instructingtermination (Step S9: No), the ultrasound observation apparatus 3returns to Step S1.

When the puncture needle detection unit 34 detects the image of thepuncture needle in the B-mode image (Step S4: Yes), and when thecomposite image generation unit 362 has not generated the compositeimage in the previous frame (Step S5: No), the ultrasound observationapparatus 3 proceeds to Step S7.

The case in which the puncture needle detection unit 34 does not detectthe image of the puncture needle in the B-mode image (Step S4: No) inStep S4 will be described. In this case, when the composite imagegeneration unit 362 has not generated the composite image in theprevious frame (Step S10: No), the ultrasound observation apparatus 3proceeds to Step S11. This situation corresponds to a situation in whichthe puncture needle detection unit 34 has never detected the punctureneedle, or a situation in which the puncture needle detection unit 34has continued to detect the image of the puncture needle up to oneprevious frame and stops detecting the image of the puncture needle atthis frame.

In Step S11, when the puncture needle information storage unit 381stores detection data of the puncture needle (Step S11: Yes), that is,when the puncture needle information storage unit 381 has stored theinformation of the point position of the image of the puncture needle ina plurality of frames up to the previous frame in succession, the motionextraction unit 35 extracts the linear motion at the point of thepuncture needle based on the history of the point position of the imageof the puncture needle (Step S12).

In Step S11, when the puncture needle information storage unit 381 doesnot store the detection data of the puncture needle (Step S11: No), theultrasound observation apparatus 3 proceeds to Step S8 described above.This situation corresponds to a situation in which the puncture needledetection unit 34 has never detected the puncture needle.

After that, the composite image generation unit 362 generates the locusof the linear motion extracted by the motion extraction unit 35, andsuperimposes the locus on the B-mode image to generate a composite image(Step S13). At this time, the composite image generation unit 362generates the composite image by performing correction to hold arelative positional relationship between the locus of the linear motionat the point of the puncture needle and the observation target of theB-mode image.

When the composite image generation unit 362 generates the locus in thisframe (Step S14: Yes), the control unit 37 deletes the detection data ofthe puncture needle stored in the puncture needle information storageunit 381 (Step S15).

Next, the display controller 371 performs control to cause the displaydevice 5 to display the composite image generated by the composite imagegeneration unit 362 (Step S16). The composite image displayed by thedisplay device 5 is, for example, the composite image 102 illustrated inFIG. 5.

After Step S16, the ultrasound observation apparatus 3 proceeds to StepS9 described above.

In Step S10, when the composite image generation unit 362 has generatedthe composite image in the previous frame (Step S10: Yes), theultrasound observation apparatus 3 proceeds to Step S13. This situationcorresponds to a situation in which undetected state of the image of thepuncture needle continues from at least one previous frame. In thissituation, in Step S13, the composite image generation unit 362generates the composite image, using the newly generated B-mode imageand the locus generated in one previous frame.

In Step S14, when the composite image generation unit 362 has notgenerated the locus in this frame (Step S14: No), that is, when thecomposite image generation unit 362 has generated the locus in a frameprior to the present frame, the ultrasound observation apparatus 3proceeds to Step S16.

According to the first embodiment described above, the composite imageis generated by extracting the linear motion at the point of thepuncture needle based on the history of the image of the puncture needlein the ultrasound image, and generating the locus of the extractedlinear motion and superimposing the locus on the ultrasound image.Therefore, the position where the puncture needle has been moved aplurality of times in the subject can be accurately grasped.

In the first embodiment, when the puncture needle detection unit 34stops detecting the image of the puncture needle, the composite imagegeneration unit 362 starts generation of the composite image, and afterthat, when the puncture needle detection unit 34 detects the image ofthe puncture needle again, the composite image generation unit 362 stopsgeneration of the composite image. Therefore, according to the firstembodiment, the user can confirm the history of the first needle biopsyduring time from when the puncture needle is taken out once to when thesecond needle biopsy is performed, and can more accurately grasp theposition where a tissue is to be collected in the second needle biopsy.

First Modification

FIG. 7 is a diagram schematically illustrating a display example of acomposite image in a first modification of the first embodiment. In thefirst modification, the composite image generation unit 362 generates acomposite image, using only loci that have passed through a commonsection a plurality of times, of linear motions extracted by the motionextraction unit 35. A composite image 103 illustrated in FIG. 7 is animage generated based on the same puncture needle information as thecomposite image 102 illustrated in FIG. 5. A locus group 131 displayedin the composite image 103 illustrates only the loci that have passedthrough a common section a plurality of times. Therefore, the number ofloci is smaller than that of the locus group 121 displayed on thecomposite image 102 generated based on the same puncture needleinformation.

According to the first modification described above, only the lociincluding an overlapping portion in a plurality of motions aredisplayed. Therefore, a portion having a high possibility of collectinga tissue can be displayed. Therefore, a user can more reliably specify aplace having a high possibility of collecting a tissue.

Note that, in the first modification, loci up to a predetermined placein descending order of the number of times of overlap may be displayed.

Second Modification

FIG. 8 is a diagram schematically illustrating a display example of acomposite image in a second modification of the first embodiment. In thesecond modification, a composite image generation unit 362 adds thenumbers of times of overlap near loci, in addition to generation of acomposite image, using only the loci having passed through a commonsection a plurality of times, of linear motions extracted by a motionextraction unit 35, similarly to the first modification. The compositeimage 104 illustrated in FIG. 8 is an image generated based on the samepuncture needle information as the composite image 102 illustrated inFIG. 5. A locus group 141 displayed in the composite image 104 displaysthe same loci as the locus group 131 of the composite image 103 and thenumber of times of overlap is displayed near each locus.

According to the second modification described above, only the lociincluding an overlapping portion in a plurality of motions is displayedtogether with the numbers of times of overlap. Therefore, the user canmore easily grasp a portion having a high possibility of collecting atissue.

Note that, in the second modification, a composite image that displayscolors and types of lines of the loci in different forms according tothe number of times of overlap may be generated instead of displayingthe number of times of overlap. Further, in the second modification,loci up to a predetermined place in descending order of the number oftimes of overlap may be displayed, similarly to the first modification.

Third Modification

FIG. 9 is a diagram schematically illustrating a display example of acomposite image in a third modification of the first embodiment. In thethird modification, the composite image generation unit 362 generates acomposite image by superimposing a region indicating a range where animage of a puncture needle has performed linear motions on a B-modeimage during time from when the puncture needle detection unit 34 startsdetecting the image of the puncture needle to when the puncture needledetection unit 34 stops detecting the image of the puncture needle. Acomposite image 105 illustrated in FIG. 9 is an image generated based onthe same puncture needle information as the composite image 102illustrated in FIG. 5. An approximately elliptical region 151illustrated in the composite image 105 is the region indicating a rangeof linear motions of the puncture needle. For example, the region 151may be set as an outermost contour of all the linear loci or may be setas an envelope surrounding all the linear loci.

According to the third modification, in a case where a user wants topuncture another region in the next biopsy motion, the user can easilygrasp the region to be newly punctured.

Note that, in a case where a puncture needle detection unit 34 detectsan image of a puncture needle again after a composite image generationunit 362 generates a composite image displaying loci, as described inthe first embodiment and first and second modifications, the compositeimage generation unit 362 may start generation of a composite image withcontour display described in the third modification. In this case, whenthe composite image generation unit 362 resumes generation of thecomposite image displaying loci when the puncture needle detection unit34 stops detecting the image of the puncture needle. The composite imagegeneration unit 362 may start generation of the composite image withcontour display when receiving an operation input of a freeze button ofan operating unit 22, and may resume generation of the composite imagedisplaying loci when receiving a re-operation input of the freezebutton.

Second Embodiment

A second embodiment is characterized in displaying a locus of an imageof a puncture needle in an ultrasound image substantially in real time.A configuration of an ultrasound observation apparatus according to thesecond embodiment is similar to that of the ultrasound observationapparatus 3 described in the first embodiment.

FIG. 10 is a flowchart illustrating an outline of processing performedby an ultrasound observation apparatus 3 according to the secondembodiment. Processing of Steps S21 to S23 sequentially corresponds tothe processing of Steps S1 to S3 described in the first embodiment.

Following Step S23, a puncture needle detection unit 34 performsprocessing of detecting an image of a puncture needle displayed in aB-mode image using the generated B-mode image (Step S24). When thepuncture needle detection unit 34 detects the image of the punctureneedle in the B-mode image (Step S24: Yes), the puncture needledetection unit 34 writes and stores information of a point position ofthe image of the detected puncture needle together with information ofdetection time and the like to a puncture needle information storageunit 381 (Step S25).

After that, when the puncture needle information storage unit 381 storesdetection data of the puncture needle (Step S26: Yes), that is, when thepuncture needle information storage unit 381 has stored the image of thepoint position of the image of the puncture needle in a plurality offrames up to a previous frame in succession, a motion extraction unit 35extracts a linear motion at the point of the puncture needle based on ahistory of the point position of the image of the puncture needle (StepS27). Meanwhile, when the puncture needle information storage unit 381does not store the puncture needle information (Step S26: No), theultrasound observation apparatus 3 proceeds to Step S30 described below.

After Step S27, a composite image generation unit 362 generates acomposite image by superimposing the locus of the linear motionextracted by the motion extraction unit 35 on an ultrasound image (StepS28).

Next, a display controller 371 performs control to cause a displaydevice 5 to display the composite image generated by the composite imagegeneration unit 362 (Step S29). FIG. 11 is a diagram schematicallyillustrating a display example of a composite image displayed by thedisplay device 5. A composite image 106 illustrated in FIG. 11 isdisplayed in a manner that a puncture needle image 111 and a locus 161of a point position of the puncture needle image 111 can be identified.FIG. 11 illustrates a case where the locus 161 is displayed by a brokenline. However, the locus 161 may be displayed in a color different fromthe puncture needle image 111 or may be displayed with a differentthickness from the puncture needle image 111.

After that, when an input unit 33 receives an input of a signalinstructing termination (Step S30: Yes), the ultrasound observationapparatus 3 terminates the series of processing. On the other hand, whenthe input unit 33 does not receive the input of a signal instructingtermination (Step S30: No), the ultrasound observation apparatus 3returns to Step S21.

A case in which the puncture needle detection unit 34 does not detectthe image of the puncture needle in the B-mode image (Step S24: No) willbe described. In this case, when the ultrasound observation apparatus 3generates the composite image in one previous frame (Step S31: Yes), thecontrol unit 37 deletes detection data of the puncture needle stored inthe puncture needle information storage unit 381 (Step S32).

After that, the display controller 371 performs control to cause thedisplay device 5 to display the B-mode image generated in Step S23 (StepS33). As described above, in the second embodiment, the locus of thepuncture needle is not displayed when the image of the puncture needleis not included in the B-mode image. After Step S33, the ultrasoundobservation apparatus 3 proceeds to Step S30.

In Step S31, when the ultrasound observation apparatus 3 has notgenerated a composite image in one previous frame (Step S31: No), theultrasound observation apparatus 3 proceeds to Sep S33.

Note that the display controller 371 may cause the display device 5 todisplay the composite image in Step S33 without performing theprocessing of Step S32. In that case, when the puncture needle is newlydetected, the display controller 371 deletes and non-displays the locusor changes the display method.

According to the second embodiment described above, the composite imageis generated by extracting the linear motion at the point of thepuncture needle based on the history of the image of the puncture needlein the ultrasound image, and generating the locus of the extractedlinear motion and superimposing the locus on the ultrasound image.Therefore, the position where the puncture needle has been moved aplurality of times in the subject can be accurately grasped.

Further, according to the second embodiment, the linear locus of theimage of the puncture needle can be grasped substantially in real time.Therefore, a position where a tissue is to be collected next can bespecified in one-time needle biopsy.

The embodiments for carrying out the present disclosure have beendescribed. However, the present disclosure should not be limited only bythe above-described first and second embodiments. For example, in thefirst and second embodiments, when the puncture needle detection unit 34starts detecting the image of the puncture needle again after stoppingdetecting the image of the puncture needle, the puncture needleinformation storage unit 381 may continuously provide and storeinformation to be newly stored and identifiable additional informationto the information of the point position of the image of the punctureneedle stored in the puncture needle information storage unit 381 sofar. In this case, the composite image generation unit 362 may generatea composite image displaying loci of the puncture needle respectivelycorresponding to the old and new information, that is, loci of thepuncture needle in different needle biopsies in an identifiable manner.

Further, as an ultrasound probe, an extracorporeal ultrasound probe thatirradiates a body surface of a subject with ultrasound waves may beapplied. The extracorporeal ultrasound probe is usually used forobserving abdominal organs (liver, gall bladder, and bladder), breast(especially mammary gland), and thyroid gland.

Additional advantages and modifications will readily occur to thoseskilled in the art. Therefore, the disclosure in its broader aspects isnot limited to the specific details and representative embodiments shownand described herein. Accordingly, various modifications may be madewithout departing from the spirit or scope of the general inventiveconcept as defined by the appended claims and their equivalents.

1. An ultrasound observation apparatus for generating an ultrasoundimage based on an ultrasound echo acquired by an ultrasound probeprovided with an ultrasound transducer that transmits an ultrasound waveto an observation target and receives an ultrasound wave reflected atthe observation target, the ultrasound observation apparatus comprising:a puncture needle detection unit configured to detect an image of apuncture needle displayed in the ultrasound image; a motion extractionunit configured to extract a linear motion at a point of the punctureneedle based on a history of the image of the puncture needle in theultrasound image; and a composite image generation unit configured togenerate a composite image by generating loci of the linear motionsextracted by the motion extraction unit and superimposing the loci onthe ultrasound image, the composite image generation unit generating thecomposite image by using the loci of the linear motions having passedthrough a common section a plurality of times among the loci of thelinear motions extracted by the motion extraction unit.
 2. Theultrasound observation apparatus according to claim 1, furthercomprising: a puncture needle information storage unit configured tostore information of a point position of the image of the punctureneedle detected by the puncture needle detection unit, wherein themotion extraction unit extracts the linear motion by acquiring a historyof the point position of the image of the puncture needle from thepuncture needle information storage unit.
 3. The ultrasound observationapparatus according to claim 1, wherein the composite image generationunit generates the composite image by generating at least a part of theloci of the linear motions extracted by the motion extraction unit andsuperimposing the part on the ultrasound image.
 4. The ultrasoundobservation apparatus according to claim 1, wherein the composite imagegeneration unit changes a display form of the loci of the linear motionsin the composite image according to number of times of passage in thecommon section.
 5. The ultrasound observation apparatus according toclaim 1, wherein the composite image generation unit generates thecomposite image in which information of number of times of passage inthe common section is added and displayed to the loci of thecorresponding linear motions.
 6. The ultrasound observation apparatusaccording to claim 1, wherein the composite image generation unitsuperimposes, on the ultrasound image, a region indicating a range ofthe linear motions performed during time from when the puncture needledetection unit starts detecting the image of the puncture needle to whenthe puncture needle detection unit stops detecting the image of thepuncture needle.
 7. The ultrasound observation apparatus according toclaim 1, wherein the composite image generation unit generates thecomposite image while holding a relative positional relationship betweenthe observation target displayed in the ultrasound image and the locusof the puncture needle.
 8. The ultrasound observation apparatusaccording to claim 1, wherein the composite image generation unit startsgeneration of the composite image when the puncture needle detectionunit stops detecting the image of the puncture needle after a state inwhich the puncture needle detection unit is detecting the image of thepuncture needle is continued.
 9. The ultrasound observation apparatusaccording to claim 8, wherein the composite image generation unit stopsgeneration of the composite image when the puncture needle detectionunit detects the image of the puncture needle in a case where thecomposite image generation unit has generated the composite image. 10.An ultrasound observation apparatus for generating an ultrasound imagebased on an ultrasound echo acquired by an ultrasound probe providedwith an ultrasound transducer that transmits an ultrasound wave to anobservation target and receives an ultrasound wave reflected at theobservation target, the ultrasound observation apparatus comprising: apuncture needle detection unit configured to detect an image of apuncture needle displayed in the ultrasound image; a motion extractionunit configured to extract a linear motion at a point of the punctureneedle based on a history of the image of the puncture needle in theultrasound image; and a composite image generation unit configured togenerate a composite image by generating loci of the linear motionsextracted by the motion extraction unit and superimposing the loci onthe ultrasound image, the composite image generation unit generating acomposite image displaying loci of the linear motions during time fromwhen the puncture needle detection unit detects the image of thepuncture needle to when the puncture needle detection unit stopsdetecting the image of the puncture needle, and loci of the linearmotions during time from when the puncture needle detection unit detectsthe image of the puncture needle again to when the puncture needledetection unit stops detecting the image of the puncture needle, indifferent display forms from each other.
 11. The ultrasound observationapparatus according to claim 1, wherein the composite image generationunit generates the composite image during when the puncture needledetection unit is detecting the image of the puncture needle.
 12. Amethod of operating an ultrasound observation apparatus for generatingan ultrasound image based on an ultrasound echo acquired by anultrasound probe provided with an ultrasound transducer that transmitsan ultrasound wave to an observation target and receives an ultrasoundwave reflected at the observation target, the method comprising:detecting, by a puncture needle detection unit, an image of a punctureneedle displayed in the ultrasound image; extracting, by a motionextraction unit, a linear motion at a point of the puncture needle basedon a history of the image of the puncture needle in the ultrasoundimage; and generating, by a composite image generation unit, a compositeimage by generating loci of the extracted linear motion andsuperimposing the loci on the ultrasound image by using only the loci ofthe linear motions having passed through a common section a plurality oftimes among the loci of the linear motions extracted by the motionextraction unit.
 13. A method of operating an ultrasound observationapparatus for generating an ultrasound image based on an ultrasound echoacquired by an ultrasound probe provided with an ultrasound transducerthat transmits an ultrasound wave to an observation target and receivesan ultrasound wave reflected at the observation target, the methodcomprising: detecting, by a puncture needle detection unit, an image ofa puncture needle displayed in the ultrasound image; extracting, by amotion extraction unit, a linear motion at a point of the punctureneedle based on a history of the image of the puncture needle in theultrasound image; and generating, by a composite image generation unit,a composite image by generating loci of the extracted linear motion andsuperimposing the loci on the ultrasound image such that loci of thelinear motions during time from when the puncture needle detection unitdetects the image of the puncture needle to when the puncture needledetection unit stops detecting the image of the puncture needle, andloci of the linear motions during time from when the puncture needledetection unit detects the image of the puncture needle again to whenthe puncture needle detection unit stops detecting the image of thepuncture needle are displayed in different display forms from eachother.
 14. A non-transitory computer-readable recording medium on whichan executable program is recorded, the program instructing a processorto execute: detecting, by a puncture needle detection unit, an image ofa puncture needle displayed in the ultrasound image; extracting, by amotion extraction unit, a linear motion at a point of the punctureneedle based on a history of the image of the puncture needle in theultrasound image; and generating, by a composite image generation unit,a composite image by generating loci of the extracted linear motion andsuperimposing the loci on the ultrasound image by using the loci of thelinear motions having passed through a common section a plurality oftimes among the loci of the linear motions extracted by the motionextraction unit.
 15. A non-transitory computer-readable recording mediumon which an executable program is recorded, the program instructing aprocessor to execute: detecting, by a puncture needle detection unit, animage of a puncture needle displayed in the ultrasound image;extracting, by a motion extraction unit, a linear motion at a point ofthe puncture needle based on a history of the image of the punctureneedle in the ultrasound image; and generating, by a composite imagegeneration unit, a composite image by generating loci of the extractedlinear motion and superimposing the loci on the ultrasound image suchthat loci of the linear motions during time from when the punctureneedle detection unit detects the image of the puncture needle to whenthe puncture needle detection unit stops detecting the image of thepuncture needle, and loci of the linear motions during time from whenthe puncture needle detection unit detects the image of the punctureneedle again to when the puncture needle detection unit stops detectingthe image of the puncture needle are displayed in different displayforms from each other.